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NASA has established a technical challenge, under the Aviation Safety Program, Vehicle 
Systems Safety Technologies project, to improve crew decision-making and response in 
complex situations. The specific objective of this challenge is to develop data and 
technologies which may increase a pilot’s (crew’s) ability to avoid, detect, and recover from 
adverse events that could otherwise result in accidents/incidents. Within this technical 
challenge, a cooperative industry-government research program has been established to 
develop innovative flight deck-based counter-measures that can improve the crew’s ability to 
avoid, detect, mitigate, and recover from unsafe loss-of-aircraft state awareness - 
specifically, the loss of attitude awareness (i.e., Spatial Disorientation, SD) or the loss-of- 
energy state awareness (LESA). A critical component of this research is to develop specific 
and quantifiable metrics which identify decision-making and the decision-making influences 
during simulation and flight testing. This paper reviews existing metrics and methods for SD 
testing and criteria for establishing visual dominance. The development of Crew State 
Monitoring technologies - eye tracking and other psychophysiological - are also discussed as 
well as emerging new metrics for identifying channelized attention and excessive pilot 
workload, both of which have been shown to contribute to SD/LESA accidents or incidents. 


I. Introduction 

A utomation plays a significant role in today’s commercial aviation system and future concepts are proposing 
increasing levels of automation and autonomy. However, “a well-trained and well-qualified pilot has been, is, 
and will be the critical center point of aircraft safety systems and an integral safety component of the entire 
commercial aviation system.” 1 Pilots serve as the last line of defense in commercial aviation today. They routinely 
apply their skill, expertise, and knowledge of an ever-changing flight context, to manage aircraft systems during a 
plethora of complex situations that may arise during a flight. This task requires effective monitoring, decision- 
making, and action to complete the flight objective while remaining cognizant of safety margin. Further, when and if 
this margin deteriorates, the pilot is responsible for changing the flight objective or circumstance such that an 
accident or incident is avoided. 

Under this backdrop, the National Aeronautics and Space Administration (NASA) has established the technical 
challenge, under the Aviation Safety Program, Vehicle Systems Safety Technologies project, to improve crew 
decision-making and response in complex situations. The specific objective of this challenge is to develop data and 
technologies which may increase a pilot’s (crew’s) ability to avoid, detect, and recover from adverse events that 
could otherwise result in accidents/incidents. 

In response to several recent accidents and incidents, NASA has initiated within this technical challenge a 
cooperative industry-government research program to develop innovative flight deck-based counter-measures that 
can improve the crew’s ability to avoid, detect, mitigate, and recover from unsafe loss-of-aircraft state awareness 
that can lead to loss of control (LOC). This work specifically targets when a pilot (crew) may experience the loss of 
attitude awareness (i.e.. Spatial Disorientation, SD) or the loss-of-energy state awareness (LESA). 

A critical component of this research will be sufficient and appropriate testing and evaluation of crew decision- 
making. The challenge is to develop specific and quantifiable metrics which identify decision-making and the 
decision-making influences which can be employed during simulation and flight testing. Metrics are critical to 
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develop appropriate and effective methods to improve decision-making. Further, metrics may provide a means of 
real-time identification and active feedback or mitigation on the flight deck. 

II. Background 

Casual and contributing factors to LOC have been analyzed (e.g.. Ref. 2) since it remains one of the largest 
contributors to fatal commercial aircraft accidents worldwide. 

Of particularly note. Bramble of the National Transportation Safety Board (NTSB) noted that “the occurrence of 
several major accidents involving flight crew SD in large commercial airplanes in the last decade marked SD as a 
phenomenon that deserves the attention of the international aviation safety community.” 2 3 Bateman 4 further 
articulated that LOC accidents hold the highest risk for fatalities. Surprisingly, Bateman’s analysis showed that 
approximately 70% of these accidents are due to SD (the “Spatial Disorientation” and “Reversion to Eastern 
Attitude Direction Indicator (ADI)” categories cited in Ref. 4) or LESA (“Low Airspeed” cited in Ref. 4). 

A. Spatial Disorientation (SD) 

SD is an incorrect perception of one’s linear and angular position and motion relative to the plane of the earth’s 
surface. In the flight environment, SD is an erroneous perception of aircraft attitude, especially in cases where there 
is limited or no external visibility, and a misperception of attitude information otherwise displayed on the flight 
instruments. 5 

SD counter-measures have been fairly well-researched especially within the military aviation domain. This 
research has primarily addressed high performance aircraft (with only a few notable exceptions), 6 because these 
vehicles are typically flown single pilot, have demanding pilot workload and mission tasking, and routinely fly at 
extreme attitudes and with acceleration capabilities that can create significant visual and motion illusions. 5 

The accident statistic data of Bramble 3 and Bateman 4 are surprising in comparison because civil transport 
category aircraft (14 Code of Federal Regulations, Part 25) adhere to rigid principles for the design of primary 
cockpit instrumentation, are operated by a crew (two pilots) who are well trained and experienced in instrument 
flight conditions, and whose maneuvering is veiy benign. One would expect that the likelihood of SD contributing to 
an accident should be extremely low. 

B. Loss-of-Energy State Awareness 

Loss of Energy State Awareness by the crew is typically characterized by a failure to monitor or understand 
energy state indications (e.g., airspeed, altitude, vertical speed, commanded thrust) and a resultant failure to 
accurately forecast the ability to maintain adequate airspeed and energy for safe flight conditions. The leading 
consequence of LESA is aircraft stall. 

Significant international research, development, and regulatory work is being performed on aircraft stall and 
upset, especially as it relates to pilot training of possible LOC/LESA in low speed approach-to-stall and stall 
conditions (e.g., see Ref. 7 for a summary). 

C. The Need for Research and Development? 

Although significant work is being or has been done for LOC due to low speed/stall and although there is an 
abundance of existing data on SD counter-measures, closer scrutiny of the data and the undercurrents and trends that 
are emerging have prompted a dedicated research and development effort for SD/LESA counter-measures. 

For instance, Bateman’s data 4 pointed out that most of the LOC accidents occur in operations outside North 
America, Australia, Japan, and Europe. Bateman hypothesized that “the lack of pilot experience and training are 
significant factors” in this statistical discrepancy. The more important question to be answered is whether these 
accidents are truly unique to operations outside of North America, Australia, Japan and Europe or are they are just 
the ‘leading edge’? Will the lack of training and experience that Bateman suspected to be a casual factor in these 
accidents catch up to the changing demographics and experience levels for the new generation of pilots entering 
service into North America, Australia, Japan and Europe service (e.g. pilots with/without military or crew resource 
management training)? Are there design or operational characteristics that exacerbate the decision-making abilities 
of these foreign crews? 

Further, the Next Generation Air Transportation System (NextGen) is going to create new operational paradigms 
which put the aircraft into tighter spacing and time constraints and may stretch the decision-making abilities for 
these crews especially due to: 

• Greater prevalence of automation. 
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• More frequent operations in close proximity to weather and terrain (e.g.. Required Navigation 

Performance and Area Navigation (RNP/RNAV)). 

• Aircraft operations closer to performance limits (e.g., optimized descent profiles). 

D. Flight Deck Counter-Measures 

NASA is undertaking a coordinated program of research, development, test, and evaluation of new flight deck- 
based counter-measures that can improve the crew’s ability to avoid, detect, mitigate, and recover from potential 
SD/LESA accidents and incidents. The goal of the counter-measures is to create: 

a) Awareness, to provide pilots with information regarding the aircraft state or hazard. 

b) Understanding and avoidance of aircraft state and hazards including the projection of the state and hazard 
conditions should the present course of action continue. 

c) Recovery skills (e.g., training), such that the knowledge, control techniques, and procedures required to 
accomplish a safe recovery when a lack of awareness or an inability to avoid or mitigate hazards has resulted in the 
onset of an aircraft upset. 


III. SD Counter-Measures 

SD research has well-established that the pilot’s vision is, by far, the most important sensory system for 
establishing and maintaining spatial orientation during flight. 5 In the SD accidents analyzed by Bateman, the loss-of- 
visual outside horizon references were cited as contributing to the pilot’s SD. 

A. Visual Dominance 

Visual dominance exists when the pilot receives through the eyes all the information used to maintain correct 
orientation. 1 Pilots are trained to recognize the importance of visual dominance and vestibular suppression but only 
through experience and proficiency can vestibular suppression be made easier - not eliminated - but easier . 8 

The term ambient vision is often used to describe this visual component of spatial orientation as it refers to the 
fact that we are not typically consciously aware of the inputs. 5 Research has established that the pilot’s ambient 
vision is, by far, the most important sensory system for establishing and maintaining spatial orientation during flight. 
This visual dominance is critical in establishing and maintaining a pilot’s spatial orientation during flight. 

Approximately 90% of the sensory inputs that a human uses for a sense of orientation are visual and of these, 
90% are peripheral vision (where peripheral vision is often defined as the vision off of the optic axis and foveal 
vision is only in the central two degrees around our direction of gaze.) 9 

Historically, the design of an aircraft’s cockpit instrumentation has been tailored to address a crew’s need to fly 
under instrument conditions including: 1) control (i.e., attitude); 2) performance (i.e., energy state); and 3) 
navigation (i.e., position relative to desired flight path). The potential for SD is considered in the design and in the 
training of pilot’s to use this information, but the display of this information is essentially a foveal, not ambient 
visual stimuli (see a state-of-the-art design in Figure 1). 
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Figure 1: Boeing 787 Flight Deck Displays 5 display units - 12-inch x 9-inch each (from Ref. 10) 


B. Primary Flight Display Test and Evaluation Metrics 

Quantitative design, test, and evaluation standards for cockpit displays to meet these instrument flight 
requirements, including spatial disorientation prevention (e.g., ‘visual dominance’) are not prevalent. Regulatory 
requirements and design standards are contained in AC25-11 for instance, but specific quantitative guidance for SD 
prevention and visual dominance are not on hand. AC25-1 1 specific guidance suggests that: 

An accurate, easy, quick-glance interpretation of attitude should be possible for all unusual 
attitude situations and other “non-normal” maneuvers sufficient to permit the pilot to 
recognize the unusual attitude and initiate an appropriate recovery within one second. 
Information to perform effective manual recovery from unusual attitudes using chevrons, 
pointers, and/or permanent ground-sky horizon on all attitude indications is recommended. 

After a rash of SD accidents, the United States Air Force faced a similar predicament in the certification of flight 
instrumentation. The approach taken was to establish a working group of subject matter experts to address the 
deficiencies in the design and development of primary flight instrumentation. 11 As part of this work, test and 
evaluation standards were created. Absolute standards were not practical but the concept that emerged was one of a 
‘comparative baseline’ (analogous to Beringer’s subsequent work of equivalent levels of performance and safety 12 ) 
(ref: Weinstein et al). The comparative baseline was a ‘standard’ head-down display instrument suite. 13 

Three instrument flight tasks were developed to test the adequacy of the primary flight instrumentation: 1) an 
Instrument Landing System (ILS) approach and landing; 2) a precision instrument control task (PICT); and, 3) 
unusual attitude recovery. A key element of these tasks were test and evaluation standards. 

• The ILS standards reflect typical approach and landing performance standards 14 

• The PICT standards emerged from pilot proficiency performance standards where the pilot’s ability to 

control the aircraft using the primary flight information is graded using exceedance or ‘bin scores’ 
criteria against established Vertical S-D maneuver criteria from aircrew standardization/evaluation 
programs, such as outlined under Air Force Instruction 11-202. 

• The unusual attitude criteria were more challenging to develop. Quantitative criteria were critical to 

allow accurate and automated processing of the data to ascertain correct or incorrect pilot recognition 
and recovery control actions. Standards for fighter type aircraft have been developed and are relatively 
straight-forward. 15 For a transport category aircraft, unusual attitude recovery criteria, shown in Table 
1, 16 are not as straight-forward. 
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Table 1. Initial Conditions And Recovery Input Scores 
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Unintentional conditions for transport category aircraft 17 are described by: 

• Pitch attitude greater than 25 deg, nose up 

• Pitch attitude greater than 10 deg, nose down 

• Bank angle greater than 45 deg 

• Within the above parameters, but flying at airspeeds inappropriate for the conditions 

The attitudes shown in Table 1 skirted the boundaries for unintentional conditions for transport category aircraft 
and yet remained within accordance with the safe operating limitations of the test aircraft. These criteria also parallel 
standards and methods used for fighter aircraft testing. 15 In creating these quantitative criteria several considerations 
were important: 

• The criteria of Table 1 show that multiple inputs can be correct. The absence of an action is sometimes 

correct. 

• A masking maneuver was flown, not intending to disorient the pilot, but to mask the attitude for recovery. 

This creates an element of startle, requiring the pilot to recognize the aircraft attitude and then effect the 
proper recovery. 

• The procedures called for the subject pilot to hold their finger tips on the controls or place their hand near 

the controls while a masking maneuver was performed. If their hand was on the controls, the subjects 
might introduce inadvertent inputs or be able to follow the masking maneuver. Once the attitude was 
reached, a tone was signaled and the subject pilot began the recovery. 

• Thresholds were applied to identify the initial inputs. Pitch, roll, and throttle inputs that first exceeded 

values of 5 lbs in pitch stick force, 3 lbs in roll stick force and 0.4 inches in throttle movement signified 
a pilot’s intentional input to recovery. These thresholds were critical to eliminate inadvertently counting 
non- significant, unintentional inputs especially as the pilot grabs the stick after the tone sounded. 

• Similar criteria have been used by others, as well as augmented by additional criteria such as whether the 

crew disconnected the auto-throttle and/or auto-pilot, maximum and minimum speed, stick shaker 
occurrence, or stall. 17 

The test methodology, such as having the pilot use fingertips on the controls, and threshold criteria can 
significantly influence the measured recovery times. Data from Gawron 16 in Figure 2 shows initial inputs for 
recovery were not always below 1.0 sec as per AC25-11 (even though the data suggest that excellent attitude 
awareness was provided by the test primary flight instrumentation). Unusual attitude recovery for an F-16 using a 
HUD format showed recoveries greater than 1.0 seconds as well, averaging about 1.4 to 1.5 seconds time to first 
input (i.e., reaction time) with 96 to 99% response accuracies. 13 

Time to recover and altitude lost have been used as UAR criteria. These criteria are logical but they are also 
problematic, especially in simulation. 
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Figure 2. Fastest Correct Reaction Time per Pilot (From Ref. 16) 


C. Flight Vs. Simulation Metrics 

Simulation is a powerful tool in research and development but its limitations must be recognized. 18 One of the 
difficulties in administering accurate, representative test and evaluation metrics for aircraft upset recoveries is the 
lack of full motion cuing in a ground-based simulator, even one as capable as a Level D simulator. These simulators 
lack the accelerations that might be encountered in entry or recovery from these extreme upset conditions. Using 
altitude loss or time-to-recover as criteria will exacerbate this deficiency. 

The work by Ward and Moreau 19 highlighted how, without ‘calibration’ to the flight environment, MD-1 l/A-300 
pilots often exceeded g-force targets for the upset recoveries, some pulling as much as 8 g’s in full-flight simulators. 
Lacking the g-cues associated with actual flight, the pilots tended to over control in an effort to expedite recoveries 
in the full-flight simulator (FFS). This led to gross control inputs and over-g conditions. (In contrast, during program 
development flight tests in the MD-10, MD-1 1, and B-747 airplanes, the same pilots performing pitch-up maneuvers 
to a 30° nose up attitude displayed the opposite effect; very slow and deliberate control inputs with very timid 
control input magnitudes, often leading to a rapid loss of energy which would have put the recovery in jeopardy 
were it allowed to continue.) Based on these observations, a digital g-meter readout was added to the HUD upset 
recovery symbology set to help guide the pilots into pulling or pushing to proper g-targets in both flight and ground- 
based simulation. In the development of a USAF HUD for primary flight, an audible g-tone was added in simulation 
(and in flight for experimental consistency) to signal to the pilots when a desirable g-level was being pulled. 15 

Unusual attitude and other testing primarily examines the ability of the pilot to use foveal visual display and 
vestibular suppression to effect a return to safe flight control conditions, since testing is usually conducted in 
simulated or actual instrument meteorological conditions (IMC). Testing of SD illusions using hexapod motion base 
platforms - typical of Level D FFS - shows that the implementation and effective pilot training of vestibular illusions 
is not feasible but the FFS can provide training for potential spatial disorientation caused by the visual illusions. 20 

One promising flight maneuver to test the potential for SD - but only possible during flight testing - is the Post- 
Roll Gillingham Illusion. 21 The test protocol, as follows, could provide an effective test of sub-threshold roll and 
visual dominance effects: 

1. Subject pilot rolls aircraft into 45 deg bank angle; coordinated flight using the primary flight display 

under IMC; 

2. Subject pilot ceases inputs; test displays are blanked; 

3. Auto-pilot/safety pilot performs roll reversal to opposite 45 deg bank angle using a pre-set roll rate; 

4. Upon reaching 45 deg opposite bank, a 1 second tone is triggered signaling pilot to regain aircraft control 

and maintain a stable bank angle; 

5. Subject pilot controls aircraft for 15 sec (or until safety pilot takes control); 

The data 21 (example in Figure 3) shows clear evidence of potential SD where the percentage of inputs by the 
pilots trended toward increased bank (wrong direction) as the roll rate (i.e., post-roll illusion) decreased toward zero. 
More data are clearly needed to identify the trend tow aid the sub-threshold roll condition from 10 to zero deg/sec. 
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Figure 3. Percentage Of Correct Roll Responses to Post-Roll Gillingham Illusion (From Ref. 21) 


D. Visual Dominance - Synthetic Vision? 

Synthetic Vision (SV) is a computer-generated image of the external scene topography that is generated from 
aircraft attitude, high-precision navigation, and data of the terrain, obstacles, cultural features, and other required 
flight information. NASA established a SV program in 1999 expressly to develop the technology for an electronic 
means of displaying in the cockpit intuitive visual-like flight references to provide significant improvements in 
terrain awareness and reductions in the potential for Controlled-Flight-lnto-Terrain incidents/accidents. As 
Bateman 4 noted, “if the pilot can see the horizon clearly outside the aircraft, the probability of loss of control is 
probably very low. The Synthetic Vision System (SVS) display does just that by bringing a ‘daytime clear visibility’ 
synthetic outside view into the cockpit ‘heads down’ into an overlay of primary flight instruments similar to those 
found on a Head-Up Display (HUD).” 

The use of an SV display, as opposed to an abstract analog of blue-over-brown, certainly seems intuitive for SD 
prevention. As a minimum, the SV display seems ideally fit to once-and-for-all resolve the outside-in, inside-out 
attitude indicator debate. 22 Changing the abstract nature of the display to be an intuitive earth-reference should 
eliminate any training effect or bias that an Eastern-trained might have and directly address the “Reversion to 
Eastern Attitude Direction Indicator” category of accidents as cited in Bateman. 4 

Beringer et al 12 evaluated the concept to determine if primary flight displays (PFDs) depicting terrain could be 
used with a level of safety equivalent to electronic attitude-direction indicators (EADIs) without terrain. Using 5 
groups of 8 pilots each in a fixed-based ground simulator, unknown (unusual) attitude recoveries were performed 
using one of three PFD concepts (EADI - baseline, blue -over-brown; full-color terrain SV; and. uniformly-brown 
terrain SV), with and without recovery guidance symbology. Performance standards (initial response time, total 
recovery time, and both initial and secondary control reversals) showed no statistically/operationally-significant 
differences across the groups. The recovery guidance received a positive response from the pilots but the 
performance data was unaffected. The most significant factor seemed to the presence of a prominent zero-pitch line 
(i.e., horizon line) that allowed the pilots to discern their attitude and to perform recoveries from unknown attitudes 
regardless of the terrain format. 

The prominence of symbology and other salient cues has been shown to be critical to effective recovery for 
cockpit instrumentation. In the evaluation of a HUD standard for the USAF, 11 the asymmetric nature of the HUD 
pitch ladder was well-received but the articulated pitch ladder lines nose-up were a significantly better awareness 
cue than the tapered nose-down pitch lines. The zenith and nadir symbols were extremely salient and provided 
instantly recognizable and actionable symbology for recovery. 

Ward and Moreau 19 evaluated specific symbology tailoring on the HUD for recoveries from extreme attitudes. 
The nominal symbology was decluttered and tailored for unusual attitude recovery, including guidance cue removal, 
changing to airspeed and altitude digital readouts with trend indicators for energy awareness, and adding AOA for 
awareness of wing loading. Interestingly, the EFVS image was also removed since “during development, it was 
noted that there were times when differentiating between cloud details and ground surface details was difficult for 
the inexperienced eye.” 

E. Visual Dominance - Peripheral Displays? 

These data suggest promise for the application of SV technology for establishing a visual analog - thus, creating 
ambient visual mechanisms for spatial orientation - but significantly more research is needed. 
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For instance, in Beringer’s work, 12 the SD display subtended approximately 16.4 deg H by 12.3 deg V, while 
portraying 30 deg laterally of the outside world (i.e., minification factor of approximately 2.0). Previc 23 noted that 
“studies have shown that little vection (visually-induced motion perception) is produced by visual stimuli within the 
central 50 degrees.” What is the field-of-view (FOV) required for visual dominance? 

The fact that visual dominance occurs readily in the real-world under normal daylight conditions provides 
guidance to synthetically creating it. Advisory material for the design of aircraft windows, specifically Advisory 
Circular (AC) 25.773, is examined. 

This advisory circular (AC) sets forth a method for demonstrating compliance with the airworthiness standards 
for transport category airplanes pertaining to pilot compartment view. (As with all ACs, the guidance material is not 
mandatory and does not constitute the only means of compliance to regulations.) This AC outlines the natural vision 
that the pilot of a commercial transport aircraft would have during visual meteorological conditions. In Figure 4, the 
visual area of the windows from the view of the Captain is shown. The data shows the out-the-window visual area as 
a function of the azimuth (left/right) from the design eye reference point, DERP. (Zero degrees azimuth is looking 
forward along the aircraft fuselage.) This data shows that the display area is greatest in the pilot’s left periphery (i.e., 
from -50 to -100 deg in azimuth). 



Figure 4: Out-the-Window Visual Information per AC25.773 

The out-the-window visual area metric provides a baseline metric, albeit a very lofty goal. 

The research to date 23 also suggests the following general requirements for synthetically-generated visual 
dominance: 

1) Wide FOV subtending at least 60 deg and preferably >100 deg in extent, with ‘correct’ spherical 

projection of the horizon; 

2) Optical scenes should be optically projected at a distance that exceeds the effective range of most 

binocular mechanisms (>5 m or preferable, larger); 

3) Temporal resolution to match the realism and believability of the imagery compared to an out-the- 

window scene (<100 msec latency); without aliasing, flicker or other optical defects; 

4) Spatial resolution is not critical in the periphery but must promote image realism (>0.5 min of arc spatial 

resolution); 

5) Sufficient content to both stimulate the ambient visual system and create the level of detail expected of a 
real-world daylight scene. The synthetic scene must believably represent the movement of Earth-fixed 
space. 

Attempts to create synthetically-derived visual dominance have been more or less successful. 

One study showed that with a very wide FOV and nearly collimated (58 deg V x 114 deg H FOV) daylight 
scene, the somatogravic illusion could be broken for pilot subjects. 24 Pilot subjects were visually coupled to the 
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scene in both simulated night and full-day scenes, breaking the somatogravic illusion. However, the data used both 
pilot and non-pilot subjects. The somatogravic illusion was not always broken for non-pilots. The study concluded 
that pilots perceive pitch rotation in a different way than non-pilots, implying differences in accommodating an 
interaction between the semi-circular canal and otolith system information. These data indicate that pilots must be 
used in any study of visual/motion perception in flight or simulated flight conditions. This work has serious 
implications for experience and training. Does the flying experience and training differences between military and 
commercially-trained pilots matter? Is SD training required for commercial transport pilots? What are the training 
recurrency requirements? 

F. Background Attitude Indicator (BAI) 

Stimulation of the peripheral, ambient vision is a precursor for visual dominance. A Peripheral Vision Display 
device was designed to take advantage of our subconscious processing of peripheral vision information for spatial 
orientation and balance. 9 The first-principles of the concept were successfully demonstrated to create visual 
dominance/vection especially in roll but the concept suffered somewhat by the limitations of the technology at the 
time. To date, large inside-out designs that span the entire cockpit (like the Peripheral Vision Device) have not 
proven to be completely successful in eliminating attitude interpretation problems, including those associated with 
roll-reversal errors. 23 

An alternate approach to create an ambient vision means of visual dominance -such as the Peripheral Vision 
Display - was the background attitude indicator (BAI). 23 During fighter aircraft-type unusual attitude recoveries, no 
statistically and operationally significant results were found in comparison between baseline (no-BAI EADI) and 
BAI conditions. The pilot subjective preference data highlighted important design considerations for the BAI such 
as horizon line depiction, texturing and coloration of the background information, and visual extent of the BAI. The 
subjective data showed that color shading and texture of the BAI provided the best visual information. These 
findings correspond nicely to Previc’s requirements for visual dominance in that there should be “sufficient content 
to both stimulate the ambient visual system and create the level of detail expected of a real-world daylight scene.” 

In the two studies from Liggett and Reising, 25 the visual extent of the BAI was limited. In the first study, a single 
8”x8” display was simulated with BAI drawn around a 6”x6.5” EADI. In a second study, the BAI was drawn across 
three simulated electronic displays, tiled side-by-side. 

The BAI concept is illustrated for a commercial transport aircraft in Figure 5. In Figure 5a, the BAI concept is 
drawn using all four display devices, like that used in Figure 1. The reference point for the SV background is the 
Captain’s EADI. 

In their second study, Liggett and Reising 25 varied the BAI reference point - i.e., whether one reference in the 
center display was used and the BAI extended across the three display units or whether a BAI was referenced to 
each display unit individually. For this single seat cockpit application, there was a preference for using the BAI 
across multiple displays to provide a continuous reference, like that of Figure 5a. 

However, should the reference be the left seat (Captain’s) ADI (Figure 5a) where the BAI matches the Captain’s 
ADI but is offset for the Co-pilot’s or instead, the center of the panel (Figure 5b) where the BAI is not offset for 
both pilot’s ADI? Or should it be split across the four panels (Figure 5c) so each ADI is properly referenced but two 
BAIs are seen? While good information, the reference point research question remains unanswered for side-by-side, 
two seat installations. 
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a) Referenced to Captain’s ADI 



b) Referenced to Center of Panel 



c) Referenced to Each ADI 



Figure 5: Background ADI Options for Two-Seat, Side-by-Side Cockpit 


In the two studies of the BAI, 25 the visual extent of the ambient information was relatively miniscule (Figure 6, 
‘BAI - Single EADI’ and ‘BAI - Tiled EADF) compared to the out-the -window view (Figure 4 - note scale change). 
The original work was trying to address the single-seat fighter type cockpit with little available instrument panel 
space so this result is understandable - a significant vection/visual dominance effect should not be expected. 

If a one-inch area around the periphery of displays like Figure 1 (or Figure 5) is used, more ambient vision may 
be stimulated (labeled ‘1” BAI New Display’ in Figure 6). But still, the visual area pales in comparison to the out- 
the -window. For reference, if the entire instrument panel like Figure 1 were used as a BAI the ambient vision 
stimulation (‘Use Entire Display Surface’ in Figure 6) is finally approaching the magnitude created by windows. 

Even if the visual extent is sufficient, questions remain as to whether this BAI concept display would be far 
enough from the DERP so that the imagery does not invoke binocular vision mechanisms (see Previc’s Criteria 2 
above). The distance is certainly not 5 m, more like 1 m. 

Further, questions remain as to the sufficiency of the synthetic scene detail (Previc’s Criteria 5 above) to create 
visual dominance. Fortunately, computer-generated imagery computing has grown tremendously for flight 
simulation application and aircraft instrumentation, enabling concepts such as Synthetic Vision. 14 Metrics are 
emerging to quantify a virtual scene 26 and its ability to promote realism or presence. The metric, termed ‘spatial 
velocity’ looks promising as a way to quantify the characteristics of scene content and optical flow as a test for 
visual dominance. 
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Figure 6. Visual Area Subtended by BAI Concepts 


IV. Channelized Attention and Crew Workload 

The preponderance of SD accidents and incidents in two-crewed airplanes suggest that there is a break-down in 
the pilot monitoring function: the pilot-flying experiences an SD-inducing event but the pilot-monitoring was unable 
or ineffective in preventing the SD event from causing an accident or incident. 

Effective monitoring and cross-checking can literally be the last line of defense, enabling a break in the chain of 
events leading to an accident or incident. One way to create effective monitoring and cross-checking is to develop 
automated methods of monitoring - essentially adding another layer of cautions and warnings driven around the 
error catching and cross-checking done by the human today. While this step may be possible, it is a formidable task 
considering the vast array of conditions, operations, and tasking that are currently covered by the pilot-monitor. 
Further, the flight deck is already littered with disparate alerts. Adding more alerts may do more harm than good . 27 

Sumwalt et al 2S identified four underlying causes of poor monitoring by the human: a) industry has not made 
pilot monitoring a primary task; b) the current system does not reward proper monitoring; c) continuous and 
effective monitoring is not natural; and d) effective monitoring leads to boredom and complacency. “While it may 
be true that humans are not naturally good monitors, performance can be significantly improved.” Although 
boredom and complacency are clearly problems with proper monitoring, the other extreme, also possibly as 
problematic, is channelized attention and excessive pilot workload. 

As described in Wickens , 29 channelized attention or attentional tunneling is the allocation of attention to a 
particular channel of information or task goal for a duration that is longer than optimal given the cost of neglecting 
events on other channels or failing to perform other tasks. The accident or incident concern is when one or both 
pilots (pilot-flying or pilot-monitoring) are attending to a task at the exclusion or omission of monitoring the pilot- 
flying or properly attending to the operation of the aircraft and/or automation. 

A. Channelized Attention Monitoring 

Instead of adding new layers of alerting, a different approach to ensure proper attentional direction might be to 
create technology that “monitors the monitor” as well as the pilot-flying. The technology identifies when effective 
monitoring isn’t occurring. 

Active and focused monitoring are vigilant behaviors that differ in the distribution of attention within an 
effective range . 30 Active monitoring refers to a wide scope of awareness, whereas focused monitoring describes a 
narrower, yet effective, scope of awareness prompted by necessary flight procedures or at times discovery of an 
anomaly. Attentional focusing becomes hazardous when it results in an individual being oblivious to all but a few 
elements in the present environment and shifting to a wider awareness is not considered. The state of absorption can 
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also be described as hypnotized, over-focused, excessive concentration, and forgetting the big picture. The cognitive 
experience has been termed attentional "tunneling" and "channelization," 1 and is similar in result to the attentional 
narrowing reported in certain high stress situations. 32 Whereas the state of absorbed attention can be desirable and 
beneficial for short term problem solving, it can be hazardous for those who monitor systems for long periods of 
time such as aircraft flight crew. 

B. Psychophysiological Monitoring 

The questions become: Can technology be used to improve pilot monitoring or identify when a pilot fixates or 
channelizes their attention to the exclusion of other tasks and information channels? 

Methods have been employed in laboratory and operational settings to track attention using non-invasive 
psychophysiological measures, including electroencephalography (EEG), electrocardiography (ECG) 33 ' 34 and 
functional near-infrared spectroscopy (fNIRS). 35,3 

The state-of-the-art use of EEG to determine various levels of operator functional state rely predominantly on 
analysis of frequency domain measures derived from raw EEG. The basis for this use of EEG is the associations of 
separable EEG frequency bandwidths, to the states of awareness or mental activity identified in the scientific 
literature. These frequency bands and their distribution across the cortex of the brain are typically used individually 
or through some formulaic comparison to define real-time or near real-time operator functional state. Current 
approaches to assess specific mental faculties such as task engagement or attention have been based on relative 
prevalence of high frequency EEG activity which tends to occur during high workload task performance. 37 ' 39 

Cardiovascular activity has been demonstrated to indicate real-time workload experienced by operators of 
complex systems. 4041 Specifically, ECG-derived indices reveal human autonomic nervous system (ANS) activity or 
arousal. The ANS employs feed-forward (from the central nervous system, or CNS, to the periphery) and feedback 
systems (from periphery to the CNS) to regulate internal bodily states. The parasympathetic nervous system (PNS) 
is the branch of the ANS controlling homeostatic or resting functions. The sympathetic nervous system (SNS) is the 
arousing (i.e., “fight or flight”) control system for the body. Assessment of heart activity provides a window into a 
predominantly SNS stress response to excess workload during acute incidents which can occur on the flight deck. 
Additionally, mental processes such as attention are reflected in the fluctuation of the activity of the ANS. 
Appropriate monitoring of a multi-task environment such as the flight deck would involve a PNS response to stress 
including attention that is flexible. 

A time series of the intervals between each consecutive heart beat can be derived from the non-invasive ECG of 
an operator while they are performing operational tasks. The “high frequency” (0.15 and 0.5 Hz) of the spectral 
analysis of ECG time series provides a measure referred to as respiratory sinus arrhythmia (RSA). RSA is a 
noninvasive, objective cardiovascular index which allows for assessment of the parasympathetic activity of the 
ANS. 42 43 Research has demonstrated that RSA serves as an effective metric of workload specifically in high 
workload conditions. 44 Furthermore, research supports the theory that in the face of environmental challenges, 
flexible parasympathetic nervous system functioning is an important component of adaptive stress regulation. 45 

fNIRS is an emerging portable brain imaging technology. In general, it provides rich potential to supplement 
behavioral data with objective neural hemodynamic measures in real-world contexts toward the identification of 
operator state. Real-time processing allows monitoring during the actual performance of real world tasks. When 
combined with real-time state classification processing, this allows for long-duration monitoring and potential use in 
brain-computer interfaces (BCI) applications. 4647 fNIRS has been applied to a train driving simulation task 
comparing manual and automated modes 48 and to the development of a robot which adapts to the multitasking state 
of a human operator. 49 Although promising, the need for improvement remains regarding the reliability of fNIRS 
measurements in operational contexts, the specificity of state detection, and the standardization of signal processing 
techniques in the research field. The great strength of fNIRS is that it can be used to detect cognitive states now only 
detectable with high fidelity function Magnetic Resonance Imaging in brain activation regions in the cortex. 50 

C. Eye Tracking 

Eye tracking provides a direct measure of a pilot’s eye gaze and possibly, attention across the flight deck. The 
basic metrics that define eye movement, fixations, and saccades are more general quantifying measures. 
Environment-specific measures such as area of interest (AOI), dwell time, and ‘link analysis’ (e.g., tying eye gaze to 
aircraft state data) provide context-specific elements that can be utilized to quantifiably assess pilot attention and 
infer pilot situation awareness. Eye tracking can be done remotely for non-invasive data collection - a capability that 
makes eye tracking very appealing to human-in-the-loop research activity. 

Utilization of gaze fixation and saccadic movement metrics have been prevalent in research for several years in 
an effort to define a global behavioral relationship between eye tracking metrics and a subject’s attention and state. 
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For example, fixation duration has been observed to correlate inversely with eye scan efficiency, and 
correspondingly efficiency has been proven to correlate inversely with workload . 51 Maximum and minimum fixation 
duration and fixation frequency are common measures that have been found to be significantly impacted by variable 
cognitive tasking. Longer fixations have been found to be indicative of increases in cognitive processing loads . 52 
Increased fixation frequency has shown to represent a more effortful search, suggesting poor performance accuracy 
and longer search times in memory tasks . 53 

Other measures such as scan path indexing and gaze entropy have also been used to characterize differences in 
eye movement behavior and associate them with specific eye movement behavior to define a state of being . 54 35 Such 
findings lead to general definitions to characterize a pilot’s state from global metrics, however, these measures leave 
out the contextual information that is derived from coupling such metrics with AOI information. This context- 
specific information is necessary to determine where and why a pilot is devoting attentional resources and to what 
degree. 

Wickens 56 suggests that there are three possible factors engaged in concurrent task management and their 
performance outcomes: confusion, cooperation, and competition. These factors may be associated with visual 
resource allocation and a pilot’s attention. Confusion is the condition where tasks that are similar interfere with the 
overall performance, while more distinct tasks have a less frequent degradation to performance. Cooperation is 
described as task processes that are similar and yield combined results. Competition of resources, particularly 
observable when driving the allocation of one resource versus another, results in a decreasing supply of resources 
for other tasks being managed. The argument is that cooperation between tasks improves when they use separate, 
instead of shared resources. 

Expanding on this theory, a contentious argument is that there is not a distinct number of resources available for 
an individual. Some believe there are multiple pools of resources as opposed to a single generic pool . 57 Regardless 
of the pools of resources being utilized - be it one or several - it is not a single source of information that is going to 
ultimately characterize an individual or a crew of individuals, but a composition of several sources of information. 
This makes any logic -based inferential information useful to predicting performance when such data is available, 
such as known task difficulty from aircraft state information and indicators of pilot attention and operator state 
derived from eye tracking. 

D. Crew State Monitoring 

Each of these psychophysiological measures and methods have strengths and limitations but the combined use of 
these measures has been suggested to provide converging indicators of a more complete assessment of operator 
state . 58 Under NASA’s SD/LESA counter-measures program, crew state monitoring research is following this work 
by integrating these technologies for real-time evaluation of the attentional state of flight crew and to identify flight 
crew work overload and channelized attention. The ultimate goal is to create methods and metrics which can 
operationally be employed to mitigate adverse crew/pilot states in their roles as the pilot monitoring or the pilot- 
flying. 


V. Concluding Remarks 

NASA has established a technical challenge, under the Aviation Safety Program, Vehicle Systems Safety 
Technologies project, to improve crew decision-making and response in complex situations. The specific objective 
of this challenge is to develop data and technologies which may increase a pilot’s (crew’s) ability to avoid, detect, 
and recover from adverse events that could otherwise result in accidents/incidents. Within this technical challenge, a 
cooperative industry-government research program has been established to develop innovative flight deck-based 
counter-measures that can improve the crew’s ability to avoid, detect, mitigate, and recover from unsafe loss-of- 
aircraft state awareness - specifically, the loss of attitude awareness (i.e.. Spatial Disorientation, SD) or the loss-of- 
energy state awareness (LESA). A critical component of this research is to develop specific and quantifiable metrics 
which identify decision-making and the decision-making influences during simulation and flight testing. One of the 
principal counter-measures to SD may be the use of new visual displays to create and establish visual dominance. 
The review of previous research and test metrics in visual dominance highlight significant gaps that need to be 
filled. Certification criteria for visual displays is non-existent for visual dominance, but some data are available for 
unusual or unintentional aircraft conditions. The development of Crew State Monitoring technologies - eye tracking 
and other psychophysiological - show promise as emerging new metrics for identifying channelized attention and 
excessive pilot workload which have been shown to contribute to SD/LESA accidents or incidents. The research to 
date indicates that combinations or fusion of the different technologies and methods will be necessary. In addition, 
these data must be merged with context-specific elements, such as area of interest (AOI) or ‘link analysis’ (e.g.. 
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tying eye gaze to aircraft state data) metrics to identify where and why a pilot is devoting attentional or workload 
resources and to what degree. 


References 

’Air Line Pilots Association. “White Paper, Safety Committee Statement of Position,” Human Factors in Aviation 
Automation, November 2008. 

“Belcastro, C.M. and Foster, J.V, “Aircraft Loss-of-Controi Accident Analysis.” AIAA Guidance, Navigation, and Control 
Conference, AIAA Paper No. 2010-8004, Toronto, Canada, August 2010. 

’’Bramble, Jr., W.J., “Spatial Disorientation Accidents in Large Commercial Airplanes: Case Studies and Countermeasures.” 
Paper presented at the 61st Annual International Aviation Safety Symposium, Flight Safety Foundation, Honolulu, HI, Oct. 2008 

4 Bateman, D., “Some Thoughts on Reducing the Risk of Aircraft Loss of Control,” AIAA Guidance, Navigation, and Control 
Conference, AIAA Paper No. 2010-8139, Toronto, Ontario, Canada, August 2010. 

5 Spatial Disorientation in Aviation, Volume 203 of Progress in Astronautics and Aeronautics, Editors: Fred H. Previc, 
William R. Ercoline. American Institute of Aeronautics and Astronautics, Reston, VA, 2004 

'’Heinle, T.E., and Ercoline, W.R., “Spatial Disorientation: Causes, Consequences and Countermeasures for the USAF,” 
paper presented at the RTO HFM Symposium on "Spatial Disorientation in Military Vehicles: Causes, Consequences and Cures", 
held in La Coruila, Spain. 15-17 April 2002, and published in NATO RTO-MP-086. 

7 Advani, S.K., Schroeder, J.A., and Burks, B„ “What Really Can Be Done in Simulation to Improve Upset Training?” AIAA 
Modeling and Simulation Technologies Conference, AIAA 2010-7791, Toronto, Canada, Aug 2010. 

instrument Flight Procedures, United States Air Force Manual (AFMAN) 11-217, Volume 1, January 2005 

9 Malcolm, R., “Pilot Disorientation and the Use of the Peripheral Vision Device,” The 1983 Annual Harry G. Armstrong 
Lecture, Aviation Space and Environment Medicine, March 1984, pp. 231-238. 

"’Jacobsen, A.R., “787 Flight Deck Displays,” Presentation at SAE G-10 Behavioral Sciences Committee Meeting, Feb. 
2010 . 

’’Weinstein, L.F., Ercoline, W.R., McKenzie, I., Bitton, D.F., and Gillingham, K.K., “Standardization Of Aircraft Control 
And Performance Symbology On The USAF Head-Up Display,” Air Force Materiel Command. Armstrong Laboratory, Report 
No. AL/CF-TR-199-0088, Nov 1993. 

l2 Beringer, D.B., Ball, J.D., Brennan, K., and Taite, S., “Comparison of a Typical Electronic Attitude-Direction Indicator 
With Terrain-Depicting Primary Flight Displays for Performing Recoveries From Unknown Attitudes: Using Difference and 
Equivalence Tests,” Department of Transportation, Federal Aviation Administration, Office of Aerospace Medicine, Report 
Number DOT/FAA/ AM-05/23, Washington, DC, Dec 2005. 

’"Hughes, T.C., Hassoun, J.A., and Barnaba, J.M., “A Comparison Of Head-Up And Head-Down Display Formats During 
Instrument Flying Tasks,” Aeronautical Systems Command, Report No.: ASC-TR-93-5003, Nov 1992. 

14 Kramer, L.J., Williams, S.P., and Bailey, R.E., “Simulation evaluation of synthetic vision as an enabling technology for 
equivalent visual operations,” Proceedings of SPIE, Vol. 6957, Enhanced and Synthetic Vision 2008, Eds: Jeff J. Gttell; Maarten 
Uijt de Haag, Orlando, FL, 2008. 

‘"Bailey, R. E., and Ohmit, E. E., “Evaluation of Proposed USAF HUD Standard” Calspan Final Report No. 7738-10, 
Contract No. F33615-88-C-3602, Buffalo, NY: Calspan Advanced Technology Center. 1991. 

16 Gawron, V. J„ and Bailey, R.E.. "In-flight Evaluation of the C-141 All-Glass Cockpit." Proceedings of the Eighth 
International Symposium on Aviation Psychology at the Ohio State University. 1995. 

17 Carbaugh D., Curnutt R„ Roberson, W., and Shikany, D. “Simulator Upset Recovery Training”. In Proceedings, of Royal 
Aeronautical Society Flight Simulation Conference Towards the Edge of the Envelope. June 2009, London. 

ls Advani, S.K., Schroeder, J.A., and Burks, B., “What Really Can Be Done in Simulation to Improve Upset Training?” paper 
presented at the AIAA Modeling and Simulation Technologies Conference, AIAA Paper No. AIAA 2010-7791, Toronto, Ontario 
Canada, 2-5 August 2010. 

19 Ward, B. and Moreau, R.D., “Advanced Maneuver and Upset Recovery Training,” Flight Safety Foundation International 
Aviation Safety Seminar, Honolulu, HI, Oct 2008. 

20 Szczepanski, C., Skibniewski, F.W., Gajsik, M., and Truszczynski, O., “Standard Flight Simulator As A Spatial 
Disorientation Training Device,” paper presented at the AIAA Modeling and Simulation Technologies Conference, AIAA 2000- 
4077, Aug 2000. 

“’Ercoline, W.R., Devilbiss, C.A., Yauch, D.W., and Brown, D.L., “Post-Roll Effects on Attitude Perception: ‘the Gillingham 
Illusion’,” Aviation Space and Environmental Medicine, Vol. 71, pp. 489-495. 

22 Previc, F., and Ercoline, W., “The outside-In attitude display concept revisited. International Journal of Aviation 
Pyschology, Vol. 9, 1999, pp. 377-401. 

“’Previc, F.H., “Visual Orientation Mechanisms” in Spatial Disorientation Eds: William Ercoline and Fred Previc; etc. 

24 Lessard, C.S., Matthews, R., and Youch, D., “Effects of Rotation on Somatogravic Illusions.” IEEE Engineering in 
Medicine and Biology. March/ April 2000. 

“"Liggett, K.K., Rcising, J.M., and Hartsock, D.C., “Development and Evaluation of a Background Attitude Indicator,” The 
International Journal of Aviation Psychology, Vol. 9, No. 1, pp. 49-71. 


14 

American Institute of Aeronautics and Astronautics 



26 So, R.H.Y., Ho, A., and Lo, W.T., “A Metric to Quantify Virtual Scene Movement for the Study of Cybersickness: 
Definition, Implementation, and Verification,” Presence: Teleoperators and Virtual Environments Vol. 10, No. 2, April 2001 pp. 
193-215 

“ 7 Dorneich, Michael C., Patricia May Ververs, and Michael D. Good. "Design and evaluation of an integrated avionics 
alerting system." Digital Avionics Systems, 2001. DASC. 20th Conference. Vol. 1. IEEE, 2001. 

28 Sumwalt, R.L., Thomas, R.J., and Dismukes, K., “Enhancing Flight-Crew Monitoring Skills Can Increase Flight Safety,” 
paper presented at the 55 lh International Air Safety Seminar, Flight Safety Foundation, Dublin, Ireland, 4-7 Nov 2002. 

29 Wickens, C.D., “Attentional Tunneling And Task Management,” paper presented at the 13th International Symposium on 
Aviation Psychology, Dayton, OH, 2005. 

30 Freeman, C. and Simmon, D., “Crew resource management model.” Flight Safety Foundation Annual International Air 
Safety Seminar. Rome, Italy., 1990. 

31 Pope, A. T., and Bogart, E. H., “Identification of Hazardous Awareness States in Monitoring Environments.” SAE 1992 
Transactions: Journal of Aerospace, Paper No. 921136, 1992. 

32 Endsley, M.R., “Toward a theory of situation awareness in dynamic systems.” Human Factors, Vol. 37, No. 1, 1995, pp. 
32-64. 

’Berka, C., Levendowski, D., Cvetinovic, M., Petrovic, M„ Davis, G., Lumicao, M., et al., “Real-time Analysis of EEG 
Indices of Alertness, Cognition, and Memory Acquired with a Wireless EEG Headset.” International Journal of Human- 
Computer Interaction, Vol. 17, No. 2, 2004. pp. 151-170. 

34 Prinzel, L.. Parasuraman, R., Freeman, F., Scerbo, M., Mikulka, P., and Pope, A., “Three experiments examining the use of 
EEG, ERPs, and HRV for real-time human-centered adaptive automation design.” NASA Langley Research Center, Hampton, 
VA, NASA Technical Publication TP-2003-212442, Jan 2003. 

33 Peck E, Lalooses F and Chauncey K., “Framing Meaningful Adaptation in a Social Context.” in ACM CHI 2011 
Workshop: Brain and Body Interfaces: Designing for Meaningful Interaction, May 2011. 

36 Izzetoglu, K., Bunce, S, Onaral, B, Pourrezaei, K and Chance, B., “Functional optical brain imaging using near-infrared 
during cognitive tasks.” International Journal of Human-Computer Interaction, Vol. 17, No. 2, 2004. pp. 21 1-227. 

37 Lorenz, B. and Parasuraman, R., “Human Operator Functional State in Automated Systems: The Role of Compensatory 
Control Strategies.” In G. R. J. Hockey, A. W. K. Gaillard & O. Burov (Eds.), Operator Functional State: The Assessment and 
Prediction of Human Performance Degradation in Complex Tasks Amsterdam: IOS Press. 2003, pp. 224-237. 

38 Tattersall, A. J., and Fairclough, S. H., “Adaptive Automation and Modes of Control.” In G. R. J. Hockey, A. W. K. 
Gaillard and O. Burov (Eds.), Operator Functional State: The Assessment and Prediction of Human Performance Degradation in 
Complex Tasks Amsterdam, Netherlands: IOS Press. 2003, pp. 238-248. 

39 Wilson, G. and Russell, C., “Operator functional state classification using multiple psychophysiological features in an air 
traffic control task.” Human Factors Vol. 45, No. 3, 2003. pp. 381-389 

40 Bryne, E. A., and Parasuraman, R. J., “Psychophysiology and adaptive automation.” Biological Psychology, Vol. 42, 1996. 
pp. 249-268. 

41 Wilson, G., “Preface to Special issue on Cardiorespiratory Measures and their Role in Studies of Performance.” Biological 
Psychology, Vol. 34, No. 2-3. 1992. pp. 91-200. 

42 Grossman, P., “Respiratory and cardiac rhythms as windows to central and autonomic biobehavioral regulation: Selection 
frames, keeping the panes clean and viewing neural topography.” Biological Psychology, Vol. 34, 1992. pp. 131-161. 

43 Porges, S. and Byrne. E., "Research methods for measurement of heart rate and respiration.” Biological Psychology, Vol. 
34, 1992.^. 93-130. 

44 Hoover, A., and Muth, E. R., “A Real-Time Index of Vagal Activity.” International Journal of Human-Computer 
Interaction, Vol. 17, No. 2, pp. 197-209. 

45 Cribbet, M., Williams, P., Gunn, H., and Rau, H., “Effects of tonic and phasic respiratory sinus arrhythmia on affective 
stress responses.” Emotion, Vol. 11, No. 1, 2011. pp. 188-93. 

46 Coyle, S., Ward, T., and Markham, C., “Brain-computer interface using a simplified functional near-infrared spectroscopy 
system.” Journal of Neural Engineering, Vol. 4, No. 3, 2007. pp. 219-226. 

47 Matthews, F.. Pearlmutter, B., Ward, T., Soraghan, C., and Markham, C., “Hemodynamics for brain-computer interfaces.” 
IEEE Signal Processing Magazine, Vol. 25, No. 1, 2008. pp. 87-94. 

48 Kojima, T.. Tsunashima, H., Shiozawa, T., Takada, H. and Sakai, T., "Measurement of train driver's brain activity by 
functional near-infrared spectroscopy (fNIRS).” Optical and Quantum Electronics, Vol. 37, No. 13-15, 2005. pp. 1319-1338. 

49 Solovey, E., Lalooses, F., Chauncey, K., Weaver, D., Parasi, M., Scheutz, M.. Sassaroli, A., Fantini, S., Schermerhorn, P.. 
Girouard, A. and Jacob, R., “Sensing Cognitive Multitasking for a Brain-Based Adaptive User Interface,” in CHI 2011, Session: 
Brain & Bio-sensor Interactions. 2011: Vancouver, BC, Canada. 2011. pp. 383-392. 

50 Cui, X.; Bray, S.; Biyant, D. M.; Glover, G. H.; Reiss, A. L. (2011). A quantitative comparison of NIRS and 1MRI across 
multiple cognitive tasks. Neuroimage Vol. 54, No. 4, pp. 2808-2821. 

51 Goldberg, J., and Kotval, X., “Eye movement-based evaluation of the computer interface.” In S. Kumar, Advances in 
Occupational Ergonomics and Safety. Amsterdam: ISO Press. 1998. pp. 529-532. 

52 Callan, D., “Eye movement relationships to excessive performance error in aviation.” Proceedings of the Human Factors 
and Ergonomics Society 42nd Annual Meeting. Santa Moica, CA: Human Factors and Ergonomics Society. 1998. pp. 1132-1136. 

5, Van Orden, Karl F., Limbert, W., Makeig, S., and Tzyy-Ping, J., "Eye activity correlates of workload during a visuospatial 
memory task." Human Factors: The Journal of the Human Factors and Ergonomics Society, Vol. 43, No. 1, 2001. pp. 111-121. 

15 

American Institute of Aeronautics and Astronautics 



54 Di Nocera, F., Terenzi, M., and Camilli, M., “Another look at scanpath: distance to nearest neighbor as a measure of mental 
workload.” In D. de Waard, K. Brookhuis, and A. Toffetti, Developments in Human Factors in Transportation, Design, and 
Evaluation. Maastricht, the Netherlands: Shaker Publishing. 2006. pp. 1-9. 

55 Gilland, J., “Driving, eye-tracking and visual entropy: exploration of age and task effects.” ProQuest, 2008. 

56 Wickens, C. D., “Processing resources in attention.” In D. L. Damos, Multiple-task performance. Washington DC: Taylor 
& Francis. 1991. pp. 3-34. 

^Marsh, R. L., Hicks, J. L., and Cook, G. I., “Focused attention on one contextual attribute does not reduce source memory 
for a different attribute.” Memory, 2004. pp. 183-192. 

58 Cowings, P.. Toscano, W., DeRoshia, C., Taylor. B., Hines, A. L., Bright, A., et al., “Converging indicators for assessing 
individual differences in adaptation to extreme environments.” Aviation, Space, and Environmental Medicine, Vol. 78, No. 5, 
2007. pp. B195-B215. 


16 

American Institute of Aeronautics and Astronautics 



